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1.  Objectives.  ^ : 

We  will  conduct  psychological  research  to  investigate  how  visual  information  is  represented  in  the  ^  \ 
human  mind  during  the  first  few  seconds  after  a  display.  We  will  investigate  (1)  how  many 
representations  exist,  when  they  are  available,  and  their  properties,  (2)  how  the  transformation  from  one 
representation  to  another  is  accomplished,  and  (3)  the  mechanisms  by  which  information  in  these 
representations  is  selected  and  retrieved.  These  issues  will  be  investigated  primarily  by  conducting 
behavioral  experiments  with  human  subjects,  in  which  the  principal  measure  is  the  time  (reaction  time, 

RT)  a  person  requires  to  respond  to  a  query  of  information  that  was  present  in  the  display.  The 
experiments  will  use  variations  of  four  different  paradigms: 

(a)  Location  Probe.  The  probe,  usually  a  visual  marker  (but  sometimes  tactile),  specifies  a  location;  the 
response  is  to  name  the  item  that  is  or  was  in  that  location. 

(b)  Identity  Probe.  The  probe,  usually  a  spoken  name  (but  sometimes  visual),  specifies  the  identity  of  a 
target  item;  the  response  is  to  name  the  item  in  the  array  to  the  right  of  the  target 

(c)  Probed  Reciting.  The  probe,  a  tone  burst,  specifies  direction;  the  response  is  to  recite  the  entire 
array  in  that  direction. 

(d)  Location-Specific  Matching.  The  probe  is  a  visual  target  item  that  also  marks  the  location  of  a  test 
item  within  the  array;  the  response  is  "yes"  or  "no",  depending  on  whether  target  and  test  items  match. 
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1.  Principal  Accomplishments 

In  our  experimental  work  we  have  been  concentrating  on  two  general  areas  within  the  proposed  research. 
The  first  is  to  investigate  the  location-probe  (LP)  and  probed-reciung  (PR)  paradigms  further,  moving 
initially  towards  tests  of  whether  the  transformations  that  underlie  the  performance  changes  with  probe 
delay  in  the  two  procedures  are  the  same  or  different.  The  second  has  been  to  test  new  procedures  for 
investigating  other  properties  of  early  and  late  representations,  especially  the  way  in  which  location 
information  is  coded.  In  both  domains  there  has  been  a  good  deal  of  software  development,  along  with 
pilot  experiments.  A  large  formal  experiment  in  the  first  area  is  in  progress. 

2.1  Relations  between  the  tranrformations  revealed  by  two  paradigms.  The  method  we  have  initially 
tested  here  is  to  compare  performance  in  control  conditions  in  which  LP  and  PR  paradigms  are  used 
separately,  with  an  experimental  LP-PR  condition  in  which  they  are  mixed  under  uncertainty,  i.e.,  in 
which,  on  any  trial,  the  subject  sees  an  array  (number  of  items  (size)  varied  from  trial  to  trial]  and  then, 
after  a  delay  (varied  from  block  to  block),  randomly  either  sees  a  location  probe  and  responds  with  the 
name  of  a  single  item,  or  hears  a  tone  and  responds  by  naming  all  the  items  in  sequence.  If  the  two 
transformations  are  the  same,  then  performance  in  the  mixed  condition  should  look  the  same  as  in  the 
connol  conditions  for  both  procedures  at  each  probe  delay,  except  possibly  for  an  effect  of  mixing  (and 
having  to  discriminate  the  two  kinds  of  probe)  on  the  intercepts  of  the  functions  that  relate  mean  RT  to 
anay  size.  Results  from  pilot  tests  thus  far  indicate  that  subjects  can  handle  the  mixed  task  smoothly; 
we  have  discovered  a  difference,  however,  between  LP  (but  not  PR)  performances  in  control  and  mixed 
conditions  (mean  RT  in  the  LP  condition  increases  substantially  more  with  array  size  in  the  LP-PR 
mixture)  indicating  that  the  two  transformations  differ.  The  full  experiment,  now  in  progress,  permits 
additional  tests,  in  which  we  compare  the  transformation  rates  across  subjects  in  the  single-task  control 
conditions,  as  assessed  by  changes  in  slopes  of  the  RT  versus  array-size  functions  they  generate  (model 
free),  and  by  fitting  serial  transformation  models  and  using  estimated  rate  parameters.  If  our  finding  is 
borne  out,  it  raises  the  question,  of  course,  of  why  the  uansformations  differ. 


Because  the  current  experiment  involved  a  mixture  of  two  tasks  in  some  conditions,  the  program  to 
generate  stimulus  materials  —  here  the  displayed  item  sequences  (and  the  probed  location  in  the  LP 
task)  —  was  entirely  rewritten.  Because  the  design  also  called  for  balancing  of  up  to  6  trial  dimensions, 
the  new  program  was  complex  (about  2000  lines  of  code  in  C).  However,  we  regard  this  major  effort  as 
worthwhile,  especially  for  experiments  in  which  subjects  are  practiced  for  many  days,  partly  because 
^jequate  balancing  they  can  learn  to  anticipate  properties  such  as  which  of  the  two  kinds  of 
response  will  be  required,  or  which  location  will  be  probed;  it  is  critical  for  for  success  of  the 
experiment  that  they  be  unable  to  anticipate  such  trial  properties. 
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2.2  Influence  of  reciting  direction  on  LP  performance  in  the  PR-LP  mixture.  We  already  know  that  the 
slope  of  the  function  that  relates  mean  RT  in  the  PR  task  to  the  number  of  digit  items  in  a  ro'v  is 
greater  when  reciting  is  from  right  to  left  (backwards)  than  forwards.  This  can  be  explained  by  the  ;dca 
that  the  transformation  process  that  precedes  the  vocal  response  depends  on  the  direction  that  reciting 
will  ultimately  take,  and,  in  particular,  that  the  order  in  which  array  items  are  transformed  is  controlled 
by  the  order  of  reciting,  and  is  hence  under  experimental  control.  If  this  is  so,  then  whether  or  not  the 
LP  and  PR  transformations  are  the  same,  we  might  see  an  effect  of  reciting  direction  on  LP  (non¬ 
reciting)  trials  that  are  mixed  randomly  with  PR  trials  in  the  LP-PR  mixture,  at  intermediate  delays.  (At 
such  delays,  only  items  early  in  the  reciting  order  would  have  been  transformed,  and  a  transformed  item 
might  produce  a  faster  LP  response.)  Thus  the  LP  performance  might  serve  as  a  sensitive  indicator  of 
the  transformation  state  of  individual  items.  Indeed  we  found  such  an  effect  when  we  compared  the 
slopes  of  the  serial-position  curves  for  LP  trials  in  LP-PR  blocks  in  which  reciting  direction  was  always 
either  forward  or  backward.  To  eliminate  biases  in  eye  position  as  a  cause,  we  developed  a  procedure  in 
which  the  required  reciting  direction  was  varied  randomly  from  trial  to  trial,  signalled  by  a  salient  visual 
cue  concurrently  with  the  brief  display:  the  effect  on  LP  trials  was  the  same.  Finally,  if  the 
transformation  time  per  item  is  increased  (by  degradation  of  the  array)  we  expected  the  left-to-right 
differential  to  increase  on  LP  trials,  and  found  indeed  that  it  did.  A  formal  experiment  to  test  for  an 
influence  of  reciting  direction  on  LP  performance  in  the  PR-LP  mixture  is  planned  for  December,  using 
subjects  currently  being  trained  in  the  mixture  condition. 

23.  Improvements  in  the  timing  of  spoken  responses.  We  have  devoted  a  good  deal  of  effort  to 
improving  the  precision  of  our  measures  of  vocal  response  latency  and  duration.  The  most  significant  of 
several  modifications  of  the  experiment-running  code  is  the  addition  of  storage  of  detailed  records  of 
subject  utterances  in  a  form  that  permits  rapid  analysis.  In  the  past,  we  have  filtered  the  speech  in 
high-frequency  and  low-frequency  bands,  and  the  energy  in  each  of  these  bands  has  been  measured  for 
each  10-msec  epoch.  A  threshold  has  been  applied  in  each  band,  and  the  utterance  is  deemed  to  have 
started  when  the  first  run  of  specified  length  of  10-msec  epochs  is  detected.  A  similar  decision  rule  has 
been  used  for  determining  when  the  utterance  ends.  (We  have  also  stored  a  measure  of  peak  level  in 
each  band.)  However,  this  method  requires  us  to  commit  ourselves  to  particular  values  of  threshold  and 
run  length  before  the  experiment,  rather  than  adjusting  these  parameters  to  particular  subjects,  based  on 
characteristics  of  their  speech.  The  alternative  of  storing  a  complete  digital  record  of  the  speech  at  a 
sampling  rate  of,  say,  lOkHz,  would  consume  too  much  space.  The  compromise  we  selected  is  to  store 
the  energy  levels  in  each  10  msec  epoch  for  each  of  the  two  bands.  This  record  will  allow  us  optimally 
to  determine  what  parameters  to  use  within  our  speech  timing  algorithm. 

2.4.  Representation  of  location  information.  A  possibility  that  has  been  discussed  a  great  deal  is  that 
one  difference  between  early  and  later  representations  of  visual  information  is  in  the  coordinate  system 
to  which  it  is  referred,  with  absolute  or  retinotopic  coordinates  early,  and  relative  or  spatiotopic 
coordinates  later.  In  earlier  work  using  the  LP  procedure  we  showed  that  the  direct-access  property 
found  for  the  early  representation  is  convincingly  demonstrated  by  both  tactile  and  visual  probes.  This 
might  seem  to  suggest  that  direct  access  is  not  contingent  on  the  use  of  absolute  coordinates,  but  the 
result  is  not  decisive  for  this  conclusion,  since  it  is  possible  that  the  response  to  a  tactile  probe  is 
mediated  by  absolute  visual  location  early,  and  by  relative  visual  location  later.  Our  principal  new 
approach  to  this  problem  is  to  probe  for  an  item  by  specifying  its  absolute  versus  relative  location  at 
dif^ferent  delays.  In  the  LP  procedure  as  used  to  date,  the  probe  combines  both  kinds  of  information. 
Absolute  location  is  represented  by  the  relation  between  the  probe  and  the  fixation  point;  Relative 
location  is  represented  by  the  relation  bciHccn  the  probe  and  the  registration  marks  which  appear  with 
the  array,  above  and  below  each  occupied  liKuiion,  and  remain  until  the  response.  In  a  relative-location 
variant,  the  registration  marks  are  removed,  and  then  reappear  in  a  (possibly)  different  set  of  contiguous 
absolute  locations,  along  with  the  probe.  In  an  absolute-location  variant,  the  registration  marks  do  not 
reappear,  so  the  probe  can  be  interpreted  only  by  using  its  location  in  absolute  space,  relative  to  the 
fixation  point  Much  to  our  surprise,  preliminary  results  indicate  that  after  practice  the  slope  of  the 
function  relating  mean  RT  to  array  size  is  invariant  over  these  three  procedures.  There  is,  however,  an 
intercept  difference,  whose  interpretation  has  to  be  considered:  If  computations  were  required  to  get 
from  one  coordinate  system  to  another,  but  their  duration  was  independent  of  array  size,  then  they  would 
be  hard  to  detect  with  our  method.)  The  suggestion  from  these  initial  data,  however,  is  that  at  long 
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delays,  relative  and  absolute  location  are  equally  useful  as  retrieval  cues.  Although  detailed  examination 
of  the  data  patterns  provides  no  basis  for  suspecting  an  effect  of  probe  type  on  the  kind  of  representation 
that  is  formed,  we  hope  to  guard  against  this  by  mixing  the  three  kinds  of  probe  within  the  same  trial 
series.  A  formal  experiment  will  be  conducted  during  December,  using  subjects  currently  being  trained. 

25  Initial  results  from  a  double-LP  procedure.  One  approach  that  may  help  clarify  the  change  in 
retrieval  mechanism  in  the  LP  procedure  is  to  present  two  probes,  simultaneously  or  with  a  small  delay, 
and  require  a  pair  of  responses,  giving  the  spoken  names  of  the  items  in  the  two  positions  probed.  Of 
special  interest  is  a  proximity  effect,  the  effect  of  the  spatial  separation  of  the  two  probes  on  the 
increment  in  retrieval  time  associated  with  the  second  item  retrieved,  which  perhaps  incorporates  a  "shift 
of  attention"  from  the  first  location  accessed  to  the  second.  We  developed  the  software  for  such  a 
procedure  and  began  data  collection,  with  very  promising  results.  It  would  be  consistent  with  inferences 
from  our  earlier  findings  with  a  single  probe  (and  the  idea  of  direct  access  by  spatial  position)  if  the 
proximity  effect  was  relatively  small  when  the  pair  of  probes  was  presented  close  in  time  to  the  array; 
this  property  of  the  representation  might  change  as  the  pair  of  probes  is  delayed,  with  the  delay 
providing  time  for  the  transformation  being  studied  to  proceed.  This  is  exactly  what  we  found:  The  two 
probes  were  separated  by  100  msec.,  and  the  array  contained  six  items.  When  the  first  probe  was 
presented  50  msec  before  array  onset,  there  was  essentially  no  effect  of  spatial  proximity:  the  average 
effect  of  increasing  the  separation  by  one  item  was  only  1  msec.  When  the  first  probe  was  presented 
650  msec  after  array  onset,  the  RT  increased  by  about  28  msec  for  each  item  between  the  two  items 
probed,  a  large  effect,  and  a  large  difference  as  caused  by  the  probe  delay.  We  found  similar  effects, 
but  less  orderly  ones,  when  the  two  probes  were  presented  simultaneously.  If  these  findings  hold  up 
then  we  may  have  learned  something  important  about  the  differences  between  initial  and  later 
representations  of  visual  information.  A  more  formal  experiment  will  be  conducted  toward  the  end  of 
November.  One  important  feature  of  this  finding  flows  from  the  fact  that  it  is  not  a  change,  with  probe 
delay,  in  the  effect  of  the  size  of  an  array,  as  are  many  of  the  effects  we  have  been  studying. 
Interpretation  of  such  array-size  effects  requires  one  to  consider  the  possible  effects  of  an  increase  with 
delay  in  the  dependence  of  memory  load  on  array  size,  as  well  as  the  possibilities  that  interest  us  more: 
a  change  in  basic  properties  of  the  visual  representation.  The  proximity  effect  is  measured  with  a  fixed 
array  size,  so  its  increase  with  delay  cannot  be  explained  in  this  way,  which  facilitates  its  interpretation. 

2.6.  Completion  of  analyses  and  publication  supporting  stages  in  mental  operations.  Basic  to  the  use  of 
RT  data  in  our  ongoing  studies  of  visual  representation  is  the  assumption  that  differences  in  RT  as  one 
varies  the  size  of  an  array,  for  example,  reflect  differences  in  just  the  encoding  and  retrieval  operations 
between  probe  and  response,  and  don’t  depend  on  details  of  the  process  by  which  the  response  is 
executed,  for  example.  The  simplest  circumstances  that  justify  such  an  assumption  are  those  in  which 
the  processes  between  probe  and  response  are  organized  into  distinct  stages,  with  one  stage  beginning 
when  and  only  when  the  prior  stage  ends.  An  important  source  of  evidence  for  such  a  stage  mechanism 
over  the  past  two  decades  has  been  the  finding  of  additive  effects  of  experimental  factors  on  mean  RT. 
In  recent  years  the  stage  model  has  been  challenged,  and  it  has  been  shown  that  there  are  other 
mechanisms  that  can  also  produce  additive  factor  effects  on  the  mean.  This  problem  led  to  the 
development  and  application  of  some  new  tests,  described  in  Publication  (1),  recently  completed.  One  of 
the  four  experiments  to  which  these  tests  were  applied  (a  flash-detection  experiment)  is  one  in  which  the 
allocation  of  attention  over  visual  space  was  investigated,  bringing  the  work  close  substantively,  as  well 
as  methodologically,  to  issues  that  are  cenu-al  to  our  re.search  objectives.  The  work  can  be  summarized 
as  follows: 

Additivity  of  the  effects  of  experimental  manipulations  on  mean  RT  has  been  taken  to  suggest  that  the 
underlying  mechanism  can  be  divided  into  independently  changeable,  serially  arranged  operations  (Stage 
Model,  additive-factor  method;  Sternberg,  1969).  A  similar  conclusion  has  been  drawn  by  Roberts 
(1987)  from  the  many  instances  of  multiplicative  effects  of  experimental  manipulations  on  response  rate 
in  animal  experiments.  In  this  paper  we  consider  two  other  explanations  of  the  same  additive  pattern  of 
mean  RTs:  a  model  with  independently  changeable  alternate  pathways,  and  the  McClelland-Ashby 
Cascade  Model.  In  all  three  models,  additivity  reflects  selectivity  of  the  influence  of  experimental 
factors  on  mental  operations.  Among  the  new  tests  described  are  comparisons  of  entire  RT  distributions. 
Applied  to  the  results  of  four  diverse  experiments  (overlapping  tasks,  classification,  identification,  and 
detection),  analyses  of  distributions  and  variances  support  the  Stage  Model  and  contradict  the  Cascade 
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and  Alternate-Pathways  models.  One  set  of  distributional  analyses,  based  on  a  suggestion  by  Ashby  and 
Townsend  (1980),  supports  the  Stage  Model  remarkably  well. 

Thus  far  five  experts  have  read  and  commented  on  a  preprint  of  the  article;  their  comments  included: 
"very  important,"  "beautiful,"  "a  wonderful  contribution,"  "like  a  fine  piece  of  art,"  and  "very 
impressive."  The  most  important  part  of  this  work  will  also  be  reported  in  Publication  (4). 

2.7.  Effects  of  two  kinds  of  degradation  on  encoding  arrays  of  characters.  Teresa  Pantzer  (a  fourth-year 
graduate  student)  and  I  have  been  engaged  in  further  analysis  of  the  effects  of  two  kinds  of  degradation 
on  data  from  the  PR  paradigm  and  in  a  paradigm  devised  and  studied  by  Pashler  &  Badgio  (1985) 
(JEPrHPP,  11,  105-121)  in  which  array-size  effects  are  studied,  also  under  conditions  that  seem  to 
require  that  all  array  items  be  encoded  before  response  initiation.  The  two  kinds  of  degradation  are 
superimposition  of  a  grid  (pattern  mask,  PM),  and  rotation  and  reflection  of  the  character  (RR).  The 
eff^ects  these  two  kinds  of  degradation  have  on  the  function  that  relates  mean  utterance  latency  to  array 
size  are  dramatically  different,  with  PM  influencing  primarily  the  intercept,  and  RR  influencing  primarily 
the  slope.  This  in  turn  suggests  that  PM  affects  a  part  of  the  encoding  process  (such  as  contour 
formation)  that  occurs  in  parallel  over  the  visual  field,  whereas  RR  affects  a  part  of  the  process  (such  as 
identification)  that  occurs  serially,  item  by  item.  This  work  will  be  reported  in  Publication  (2),  recently 
drafted. 

2.8.  Effects  of  legibility  on  order  of  processing.  One  possibility  that  has  never  been  seriously  tested,  to 
our  knowledge,  is  that  the  relative  legibility  of  displayed  items  has  an  effect  on  the  order  in  which  they 
are  searched  (in  a  search  task)  or  otherwise  processed.  This  is  clearly  important  in  studies  such  as  the 
one  described  in  Section  2.7  above,  in  which  legibility  is  explicitly  manipulated.  With  Peter  Marvit,  a 
first-year  graduate  student,  I  have  begun  planning  some  experiments  to  investigate  this  question,  using 
the  same  two  kinds  of  degradation  to  manipulate  legibility  as  in  the  study  described  above.  It  seems 
possible  that  PM  will  influence  processing  order  while  RR  will  not,  just  because  PM  influences  a 
parallel  component  of  the  encoding  process  that  precedes  and  influences  a  serial  component,  whereas  RR 
controls  a  serial  component  that  requires  an  advance  commitment  to  order. 
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